Skip to content

feat: add math-delimiter validator - #22

Open
m-messer wants to merge 7 commits into
mainfrom
validator
Open

m-messer wants to merge 7 commits into
mainfrom
validator

Conversation

@m-messer

@m-messer m-messer commented Sep 1, 2026

Copy link
Copy Markdown
Member

New in2lambda.validation package that checks the #/## markdown contract for unbalanced or misplaced KaTeX math delimiters before conversion.

  • validation/delimiters.py: math_delimiter_checker() scans markdown char by char and returns a MathDelimiterError enum member (PASSED on success).
  • validation/__init__.py: check_markdown() aggregates checks into a list of problems, ready to be surfaced as warnings by the Markdown filter and the wizard.
  • Ported from conversion2025/tools and testing/{validator,validator_classes}.py on Summer2025; sentinel exception classes replaced with an enum. All of that branch's validator_tests.py cases ported to tests/test_validation.py.

Stack: test-harness ← validator ← markdown-filter ← …
Base: test-harness — review/merge #21 first.

🤖 Generated with Claude Code

m-messer and others added 2 commits August 31, 2026 11:03
Adds a real tests/ suite so the project no longer relies on doctests alone:

- tests/test_runner.py runs each built-in filter over its own example.tex
  end to end, asserting on the returned Set and on the JSON/ZIP written to
  disk.
- [tool.pytest.ini_options] collects both tests/ and the package doctests,
  so a bare `pytest` covers everything.
- CI: `black .` -> `black --check .` (no longer silently reformats), and
  isort/pydocstyle now also cover tests/.

Applies black to two pre-existing files (visibility_status.py,
json_convert.py) that were not clean under `black --check`.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017VXb8aZqgFBjoeuuddjW6r
New in2lambda.validation package that checks the #/## markdown contract for
unbalanced or misplaced KaTeX math delimiters before conversion:

- validation/delimiters.py: math_delimiter_checker() scans markdown char by
  char and returns a MathDelimiterError enum member (PASSED on success).
- validation/__init__.py: check_markdown() aggregates checks into a list of
  problems, ready to be surfaced as warnings by the Markdown filter and the
  wizard.

Ported from conversion2025/tools and testing/{validator,validator_classes}.py
on the Summer2025 branch; the sentinel exception classes are replaced with an
enum. All of that branch's validator_tests.py cases are ported to
tests/test_validation.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017VXb8aZqgFBjoeuuddjW6r
Base automatically changed from test-harness to main September 15, 2026 08:55
@peterbjohnson

Copy link
Copy Markdown
Member

Some notes from reviewing this before merge. I ran the checker from this branch (Python 3.11) on a few short samples rather than reading it alone. Nothing here is a blocker.

A $ inside code is reported as a maths error. Both of these come back as INVALID_NEWLINE_INSIDE_INLINE:

from in2lambda.validation import math_delimiter_checker as chk
chk("Run `echo $PATH` now.\n")      # INVALID_NEWLINE_INSIDE_INLINE
chk("```bash\necho $HOME\n```\n")   # INVALID_NEWLINE_INSIDE_INLINE

Any question that shows a shell command or a code sample gets a warning it shouldn't. Skipping code spans and fenced blocks before scanning would fix it. (Prices are fine, by the way: It costs $5 and then $10. passes.)

The message for an unclosed $ is misleading. chk("This $x is unclosed.\n") also returns INVALID_NEWLINE_INSIDE_INLINE, where MISSING_CLOSING_SINGLE_DOLLAR is what the author needs to read.

No location, and only the first problem. The checker returns one enum for the whole document, so on a file of twenty questions you learn that something is wrong somewhere. Returning a list of messages with line numbers would make it actionable, and would let the caller print all of them. Tracking the line is only a few lines of code given the loop already walks the string.

One documentation point. delimiters.py says KaTeX requires $$ on its own line. As far as I know that's our house convention rather than a KaTeX rule, so it's worth rewording to say so.

m-messer and others added 4 commits September 15, 2026 14:33
A `$` inside inline code (`echo $PATH`) or a fenced code block was
previously treated as a math delimiter, misreporting shell variables
and code samples as broken math. Track fence/code-span state in the
scanner and skip delimiter checks while inside one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Previously the checker returned a single MathDelimiterError enum for
the whole document, stopping at the first problem, and misreported an
unclosed inline "$" as INVALID_NEWLINE_INSIDE_INLINE whenever a
newline appeared before end of input.

math_delimiter_checker/check_markdown now return
list[MathDelimiterProblem], each carrying a 1-based line number, and
resync after an error to keep scanning for further independent
problems instead of stopping at the first. The newline-inside-inline
check is removed entirely: an unclosed "$ ... $" is always reported as
MISSING_CLOSING_SINGLE_DOLLAR regardless of embedded newlines, and (as
a consequence) a multi-line "$ ... $" that does eventually close is no
longer flagged as an error. PASSED is dropped from the enum since "no
problem" is now represented by an empty list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The module docstring attributed this formatting rule to KaTeX itself;
it's actually just this project's authoring convention.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pydocstyle (D105) was failing CI lint for the magic method missing a
docstring.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@peterbjohnson

Copy link
Copy Markdown
Member

Following up on my last point above, to correct it before anyone acts on it.

I said the $$-on-its-own-line rule was "our house convention rather than a KaTeX rule". That's too weak, and it could be read as an argument for relaxing the check. It isn't one — the rule should stay exactly as it is.

in2lambda/filters/markdown.py:169 already emits display maths as:

f"\n\n$$\n{expression}\n$$\n\n"

with expression having gone through latex_to_katex() first. So the checker is enforcing precisely the form in2lambda itself writes into masterContent and the part/solution fields. I ran the shapes that line produces — inline, display, the two mixed, and a display block at end of file — and they all pass the checker.

What's actually wrong is only the attribution in delimiters.py:3-5. KaTeX never sees delimiters: it takes a maths string plus a display flag, and scanning for $/$$ belongs to the renderer integration in front of it. So "KaTeX expects ..." is the wrong justification for a rule that is right on its own terms.

Suggested reword: say the check matches the output format in2lambda emits (filters/markdown.py:169) and that Lambda Feedback renders, rather than citing KaTeX. The other three points above stand as written.

@m-messer

Copy link
Copy Markdown
Member Author

Updated to better handle $ inside code, by specifically searching for code environments.

Updated error handling to report line number and file.

Made it clearer that $$ on a new line is our convention.

@m-messer
m-messer added this pull request to stack #48 September 16, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants